51 research outputs found
Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks
Taking a photo outside, can we predict the immediate future, e.g., how would
the cloud move in the sky? We address this problem by presenting a generative
adversarial network (GAN) based two-stage approach to generating realistic
time-lapse videos of high resolution. Given the first frame, our model learns
to generate long-term future frames. The first stage generates videos of
realistic contents for each frame. The second stage refines the generated video
from the first stage by enforcing it to be closer to real videos with regard to
motion dynamics. To further encourage vivid motion in the final generated
video, Gram matrix is employed to model the motion more precisely. We build a
large scale time-lapse dataset, and test our approach on this new dataset.
Using our model, we are able to generate realistic videos of up to resolution for 32 frames. Quantitative and qualitative experiment results
have demonstrated the superiority of our model over the state-of-the-art
models.Comment: To appear in Proceedings of CVPR 201
One-Shot Relational Learning for Knowledge Graphs
Knowledge graphs (KGs) are the key components of various natural language
processing applications. To further expand KGs' coverage, previous studies on
knowledge graph completion usually require a large number of training instances
for each relation. However, we observe that long-tail relations are actually
more common in KGs and those newly added relations often do not have many known
triples for training. In this work, we aim at predicting new facts under a
challenging setting where only one training instance is available. We propose a
one-shot relational learning framework, which utilizes the knowledge extracted
by embedding models and learns a matching metric by considering both the
learned embeddings and one-hop graph structures. Empirically, our model yields
considerable performance improvements over existing embedding models, and also
eliminates the need of re-training the embedding models when dealing with newly
added relations.Comment: EMNLP 201
Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation
Existing research studies on vision and language grounding for robot
navigation focus on improving model-free deep reinforcement learning (DRL)
models in synthetic environments. However, model-free DRL models do not
consider the dynamics in the real-world environments, and they often fail to
generalize to new scenes. In this paper, we take a radical approach to bridge
the gap between synthetic studies and real-world practices---We propose a
novel, planned-ahead hybrid reinforcement learning model that combines
model-free and model-based reinforcement learning to solve a real-world
vision-language navigation task. Our look-ahead module tightly integrates a
look-ahead policy model with an environment model that predicts the next state
and the reward. Experimental results suggest that our proposed method
significantly outperforms the baselines and achieves the best on the real-world
Room-to-Room dataset. Moreover, our scalable method is more generalizable when
transferring to unseen environments.Comment: 21 pages, 7 figures, with supplementary materia
- …